1. Field of the Invention
This invention relates to adaptive filters employing a normalized time domain least mean square (LMS) algorithm for applications where the statistics of the input processes are unknown or changing including noise cancellation, line enhancing, and adaptive array processing. More particularly, this invention relates to adaptive filters employing a normalized time domain least mean square algorithm having enhanced convergence rates by virtue of the use of an update gain greater than that accepted in the art as being an upper bound.
2. Description of Related Art
FIG. 1 shows a block diagram of a classic system identification adaptive filter configuration. The objective of an adaptive filter 10 is to change, or adapt, the coefficients of the filter to match as closely as possible the response of an unknown system 11. The unknown system and the adaptive filter process the same input signal x(k) and have outputs y(k), the desired signal, and y′(k). The adaptive filter 10 is adapted using a normalized time domain least mean square algorithm. First, the error signal, e(k), which is the difference between the output y′(k) of the adaptive filter and the output y(k) of the unknown system is computed. Using the calculated error as a basis, the adaptive filter changes its coefficients in an attempt to reduce the error.
In much of the research of the least mean square algorithm, the update gain has been referred to as variable step-size (See Harris, Richard W. et al., “A Variable Step (VS) Adaptive Filter Algorithm”, IEEE Trans. Acoust. Speech. Signal Processing, vol. ASSP-34, pp. 309-316, April 1986; Aboulnasr, Tyseer et al., “A Robust Variable Step-Size LMS-Type Algortihm: Analysis and Simulations”, IEEE Trans. Signal Processing, vol. 45, pp. 631-639, March 1997; Pazaitis, Dimitrios I. et al., “A Novel Kurtosis Driven Variable Step-Size Adaptive Algorithm”, IEEE Trans. Signal Processing, vol. 47, pp 864-872, March 1999; Evans, Joseph B. et al., “Analysis and Implementation of Variable Step Size Adaptive Algorithms”, IEEE Trans. Signal Processing, vol. 41, pp. 2517-2535, August 1993; Kwong, Raymond H. et al., “A Variable Step Size LMS Algorithm”, IEEE Trans. Signal Processing, vol. 40, pp 1633-1642, July 1992; Gelfand, Saul B. et al., “The Stability of Variable Step-Size LMS Algorithms”, IEEE Trans. Signal Processing, vol. 47, pp. 3277-3288, December 1999.); variable convergence rate (See Diniz, Paulo S. R. et al., “Analysis of LMS-Newton Adaptive Filtering Algorithms with Variable Convergence Factor”, IEEE Trans. Signal Processing, vol. 43, pp. 617-627, March 1995); and even multistep size (See Soo, Jia-Sien et al., “A Multistep Size (MSS) Frequency Domain Adaptive Filter”, IEEE Trans. Signal Processing, vol. 39, pp. 115-121, January 1991). These terms are not quite accurate.
Most of the research does not differentiate between a variable update gain and a non-uniform update gain. The scalar form of the LMS is well known ash(x+1)=h(k)+μe(k)x(k)with μ equal to scalar update gain, e(k) the error at time k, and x(k)=[x(k), x(k−1), . . . , x(k−N+1)]T is an N-dimensional column vector at time k. The scalar value for μ results in a constant, uniform update gain. Thus, the gain, μ, can be replaced with the matrix
  M  =      [                            μ                                                                          0                                                                                      ⋱                                                                                      0                                                                          μ                      ]  resulting in a LMS algorithmh(x+1)=h(k)+Me(k)x(k)In this case, the update gain is constant and uniform across all coefficients of the vector h(k).
An example of a variable step size with uniform update gain is the normalized LMS algorithm (NLMS)
      h    ⁡          (              k        +        1            )        =            h      ⁡              (        k        )              +                  μ        ⁢                                  ⁢                  e          ⁡                      (            k            )                          ⁢                  x          ⁡                      (            k            )                                                            x            T                    ⁡                      (            k            )                          ⁢                  x          ⁡                      (            k            )                              The effective update gain is
  μ                    x        T            ⁡              (        k        )              ⁢          x      ⁡              (        k        )            and varies with σx2, the variance of the input. Although the step size is variable, the update gain is still uniform across all coefficients of h.
In some research, the update gain matrix M has been replaced by
  M  =      [                                        μ            1                                                                                    0                                                                                      ⋱                                                                                      0                                                                                      μ            N                                ]  with each μi not necessarily equal to form a non-uniform gain matrix. Publications of some performance characteristics for this form state the upper bound for μi of M generally as
            μ      i        ≤          2              λ        max              or            μ      i        ≤          2              3        ⁢                  tr          ⁡                      [            R            ]                              While these are upper bounds, they are generally not good upper bounds. Improvements in convergence speed for the NLMS algorithm has been sought by a number of researchers. However, the researchers have generally been limited by the criterion that the maximum limit for the update gain is as specified above.